THE  INTERNATIONAL 


C2  JOURNAL 


VOLUME  1,  NUMBER  2,  2007 


SPECIAL  ISSUE 


Decision  Support  for  Network-Centric 
Command  and  Control 


GUEST  EDITOR 


Sandeep  Mulgund 

The  MITRE  Corporation 


Adaptive  Automation  for  Human-Robot  Teaming 
in  Future  Command  and  Control  Systems 

Raja  Parasuraman 
Michael  Barnes 
Keryl  Cosenzo 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

2QQ7  2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2007  to  00-00-2007 

4.  TITLE  AND  SUBTITLE 

Adaptive  Automation  for  Human-Robot  Teaming  in  Future  Command 
and  Control  Systems 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Army  Research  Laboratory, Human  Research  &  Engineering 

Directorate, Aberdeen  Proving  Ground, MD, 21005 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

see  report 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

_ _ _  ABSTRACT 

18.  NUMBER  19a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  OS 

unclassified  unclassified  unclassified  Report  (SAR) 

30 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


THE  INTERNATIONAL  C2  JOURNAL 


David  S.  Alberts,  Chairman  of  the  Editorial  Board,  OASD-NII,  CCRP 
Joseph  R.  Lewis,  Managing  Editor 

The  Editorial  Board 

Berndt  Brehmer  (SWE),  Swedish  National  Defence  College 
Reiner  Huber  (GER),  Universitaet  der  Bundeswehr  Muenchen 
Viggo  Lemche  (DEN),  Danish  Defence  Acquisition  and  Logistics  Organization 
James  Moffat  (UK),  Defence  Science  and  Technology  Laboratory  (DSTL) 

Mark  Nissen  (USA),  Naval  Postgraduate  School 

Ross  Pigeau  (CAN),  Defence  Research  and  Development  Canada  (DRDC) 

Mink  Spaans  (NED),  TNO  Defence,  Security  and  Safety 

Associate  Editors 

Gerard  Christman,  U.S.  OSD  Technical  Services  -  Femme  Comp  Inc. 

R.  Scott  Cost,  Johns  Hopkins  University  Applied  Physics  Laboratory 

Raymond  J.  Curts,  Strategic  Consulting,  Inc 

Paul  K.  Davis,  RAND  Corporation 

Petra  M.  Eggenhofer,  Munich  Bundeswehr  University 

Elliot  Entin,  Aptima 

Stuart  Grant,  Defence  Research  and  Development  Canada  (DRDC) 

Juergen  Grosche,  FGAN-FKIE,  Germany 
Paul  Labbe,  Defence  Research  and  Development  Canada 
Michael  Malm,  Swedish  Defence  Research  Agency 
Sandeep  Mulgund,  The  MITRE  Corporation 
Philip  W.  Pratt,  Northrop  Grumman 

Jens  Roemer,  Fachschule  der  Bundeswehr fir  Informatiktechnik 

Pamela  Savage-Knepshield,U5.  Army  Research  Laboratory,  Human  Research  & 

Engineering  Directorate 

Keith  Stewart,  Defence  Research  and  Development  Canada  (DRDC) 

Andreas  Tolk,  Old  Dominion  University 


About  the  Journal 


The  International  C2  Journal  was  created  in  2006  at  the  urging  of  an  inter¬ 
national  group  of  command  and  control  professionals  including  individuals 
from  academia,  industry,  government,  and  the  military.  The  Command 
and  Control  Research  Program  (CCRP,  of  the  U.S.  Office  of  the  Assistant 
Secretary  of  Defense  for  Networks  and  Information  Integration,  or  OASD- 
NII)  responded  to  this  need  by  bringing  together  interested  professionals  to 
shape  the  purpose  and  guide  the  execution  of  such  a  journal.  Today,  the 
Journal  is  overseen  by  an  Editorial  Board  comprising  representatives  from 
many  nations. 

Opinions,  conclusions,  and  recommendations  expressed  or  implied  within 
are  solely  those  of  the  authors.  They  do  not  necessarily  represent  the  views  of 
the  Department  of  Defense,  or  any  other  U.S.  Government  agency. 

Rights  and  Permissions:  All  articles  published  in  the  International  C2 
Journal  remain  the  intellectual  property  of  the  authors  and  may  not  be  dis¬ 
tributed  or  sold  without  the  express  written  consent  of  the  authors. 


For  more  information 


Visit  us  online  at:  www.dodccrp.org 


Contact  our  staff  at:  publications@dodccrp.org 


The  International  C2  Journal  [  Vol  1,  No  2  |  43-68 


Adaptive  Automation  for 
Human-Robot  Teaming  in  Future 
Command  and  Control  Systems 


Raja  Parasuraman  (George  Mason  University) 
Michael  Barnes  (Army  Research  Laboratory) 
Keryl  Cosenzo  (Army  Research  Laboratory) 


Abstract 

Advanced  command  and  control  (C2)  systems  such  as  the  U.S. 
Army’s  Future  Combat  Systems  (FCS)  will  increasingly  use  more 
flexible,  reconfigurable  components  including  numerous  robotic 
(uninhabited)  air  and  ground  vehicles.  Human  operators  will  be 
involved  in  supervisory  control  of  uninhabited  vehicles  (UVs)  with 
the  need  for  occasional  manual  intervention.  This  paper  discusses 
the  design  of  automation  support  in  C2  systems  with  multiple  UVs. 
Following  a  model  of  effective  human-automation  interaction 
design  (Parasuraman  et  al.  2000),  we  propose  that  operators  can 
best  be  supported  by  high-level  automation  of  information  acquisi¬ 
tion  and  analysis  functions.  Automation  of  decisionmaking  func¬ 
tions,  on  the  other  hand,  should  be  set  at  a  moderate  level,  unless 
1 00  percent  reliability  can  be  assured.  The  use  of  adaptive  automa¬ 
tion  support  technologies  is  also  discussed.  We  present  a  framework 
for  adaptive  and  adaptable  processes  as  methods  that  can  enhance 
human-system  performance  while  avoiding  some  of  the  common 
pitfalls  of  “static”  automation  such  as  over-reliance,  skill  degrada¬ 
tion,  and  reduced  situation  awareness.  Adaptive  automation  invoca¬ 
tion  processes  are  based  on  critical  mission  events,  operator 
modeling,  and  real-time  operator  performance  and  physiological 
assessment,  or  hybrid  combinations  of  these  methods.  We  describe 
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the  results  of  human-in-the-loop  experiments  involving  human 
operator  supervision  of  multiple  UVs  under  multi-task  conditions  in 
simulations  of  reconnaissance  missions.  The  results  support  the  use 
of  adaptive  automation  to  enhance  human-system  performance  in 
supervision  of  multiple  UVs,  balance  operator  workload,  and 
enhance  situation  awareness.  Implications  for  the  design  and  field¬ 
ing  of  adaptive  automation  architectures  for  C2  systems  involving 
UVs  are  discussed. 


Introduction 

Unmanned  air  and  ground  vehicles  are  an  integral  part  of 
advanced  command  and  control  (C2)  systems.  In  the  U.S.  Army’s 
Future  Combat  Systems  (FCS),  for  example,  uninhabited  vehicles 
(UVs)  will  be  an  essential  part  of  the  future  force  because  they  can 
extend  manned  capabilities,  act  as  force  multipliers,  and  most 
importantly,  save  lives  (Barnes  et  al.  2006).  The  human  operators  of 
these  systems  will  be  involved  in  supervisory  control  of  semi-autono¬ 
mous  UVs  with  the  need  for  occasional  manual  intervention.  In  the 
extreme  case,  soldiers  will  control  UVs  while  on  the  move  and  while 
under  enemy  fire. 

All  levels  of  the  command  structure  will  use  robotic  assets  such  as 
UVs  and  the  information  they  provide.  Control  of  these  assets  will 
no  longer  be  the  responsibility  of  a  few  specially  trained  soldiers  but 
the  responsibility  of  many.  As  a  result,  the  addition  of  UVs  can  be 
considered  a  burden  on  the  soldier  if  not  integrated  appropriately 
into  the  system.  Workload  and  stress  will  be  variable  and  unpredict¬ 
able,  changing  rapidly  as  a  function  of  the  military  environment. 
Because  of  the  likely  increase  in  the  cognitive  workload  demands  on 
the  soldier,  automation  will  be  needed  to  support  timely  decision¬ 
making.  For  example,  sensor  fusion  systems  and  automated  decision 
aids  may  allow  tactical  decisions  to  be  made  more  rapidly,  thereby 
shortening  the  “sensor-to-shooter”  loop  (Adams  2001;  Rovira  et  al. 
2007).  Automation  support  will  also  be  mandated  because  of  the 
high  cognitive  workload  involved  in  supervising  multiple  unmanned 
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air  combat  vehicles  (for  an  example  involving  tactical  Tomahawk 
missiles,  see  Cummings  and  Guerlain  2007). 

The  automation  of  basic  control  functions  such  as  avionics,  collision 
avoidance,  and  path  planning  has  been  extensively  studied  and  as  a 
result  has  not  posed  major  design  challenges  (although  there  is  still 
room  for  improvement).  How  information-gathering  and  decision¬ 
making  functions  should  be  automated  is  less  well  understood. 
However,  the  knowledge  gap  has  narrowed  in  recent  years  as  more 
research  has  been  conducted  on  human-automation  interaction 
(Billings  1997;  Parasuraman  and  Riley  1997;  Sarter  et  al.  1997; 
Wiener  and  Curry  1980).  In  particular,  Parasuraman  et  al.  (2000) 
proposed  a  model  for  the  design  of  automated  systems  in  which 
automation  is  differentially  applied  at  different  levels  (from  low,  or 
fully  manual  operation,  to  high,  or  fully  autonomous  machine  oper¬ 
ation)  to  different  types  or  stages  of  information-processing  func¬ 
tions.  In  this  paper  we  apply  this  model  to  identify  automation  types 
best  suited  to  support  operators  in  C2  systems  involved  in  interact¬ 
ing  with  multiple  UVs  and  other  assets  during  multi-tasking  mis¬ 
sions  under  time  pressure  and  stress.  We  describe  the  results  of  two 
experiments.  First,  we  provide  a  validation  of  the  automation  model 
in  a  study  of  simulated  C2  involving  a  battlefield  engagement  task. 
This  simulation  study  did  not  involve  UVs,  but  multi -UV  supervi¬ 
sion  is  examined  in  a  second  study. 

We  propose  that  automation  of  early-stage  functions — information 
acquisition  and  analysis — can,  if  necessary,  be  pursued  to  a  very 
high  level  and  provide  effective  support  of  operators  in  C2  systems. 
On  the  other  hand,  automation  of  decisionmaking  functions  should 
be  set  at  a  moderate  level  unless  very  high-reliability  decision  algo¬ 
rithms  can  be  assured,  which  is  rarely  the  case.  Decision  aids  that 
are  not  perfectly  reliable  or  sufficiently  robust  under  different  opera¬ 
tional  contexts  are  referred  to  as  imperfect  automation.  The  effects  on 
human-system  performance  of  automation  imperfection — such  as 
incorrect  recommendations,  missed  alerts,  or  false  alarms — must  be 
considered  in  evaluating  what  level  of  automation  to  implement 
(Parasuraman  and  Riley  1997;  Wickens  and  Dixon  2007).  We  also 
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propose  that  the  level  and  type  of  automation  can  be  varied  during 
system  operations — so-called  adaptive  or  adaptable  automation.  A 
framework  for  adaptive  and  adaptable  processes  based  on  different 
automation  invocation  methods  is  presented.  We  describe  how 
adaptive/ adaptable  automation  can  enhance  human-system  perfor¬ 
mance  while  avoiding  some  of  the  common  pitfalls  of  “static”  auto¬ 
mation  such  as  operator  over-reliance,  skill  degradation,  and 
reduced  situation  awareness.  Studies  of  human  operators  supervis¬ 
ing  multiple  UVs  are  described  to  provide  empirical  support  for  the 
efficacy  of  adaptive/ adaptable  automation.  We  conclude  with  a  dis¬ 
cussion  of  adaptive  automation  architectures  that  might  be  incorpo¬ 
rated  into  future  C2  systems  such  as  FCS.  While  our  analysis  and 
empirical  evidence  is  focused  on  systems  involving  multiple  UVs, 
the  conclusions  have  implications  for  C2  systems  in  general. 


Automation  of  Information  Acquisition  and  Analysis 

Many  forms  of  UVs  are  being  introduced  into  future  C2  systems  in 
an  effort  to  transform  the  modern  battle  space.  One  goal  is  to  have 
component  robotic  assets  be  as  autonomous  as  possible.  This  requires 
both  rapid  response  capabilities  and  intelligence  built  into  the  system. 
However,  ultimate  responsibility  for  system  outcomes  always  resides 
with  the  human;  and  in  practice,  even  highly  automated  systems  usu¬ 
ally  have  some  degree  of  human  supervisory  control.  Particularly  in 
combat,  some  oversight  and  the  capability  to  override  and  control 
lethal  systems  will  always  be  a  human  responsibility  for  the  reasons  of 
system  safety,  changes  in  the  commander’s  goals,  and  avoidance  of 
fratricide,  as  well  as  to  cope  with  unanticipated  events  that  cannot  be 
handled  by  automation.  This  necessarily  means  that  the  highest  level 
of  automation  (Sheridan  and  Verplank  1978)  can  rarely  be  achieved 
except  for  simple  control  functions. 

The  critical  design  issue  thus  becomes:  What  should  the  level  and 
type  of  automation  be  for  effective  support  of  the  operator  in  such 
systems  (Parasuraman  et  al.  2000)?  Unfortunately,  automated  aids 
have  not  always  enhanced  system  performance,  primarily  due  to 
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problems  in  their  use  by  human  operators  or  to  unanticipated  inter¬ 
actions  with  other  sub-systems.  Problems  in  human-automation 
interaction  have  included  unbalanced  mental  workload,  reduced 
situation  awareness,  decision  biases,  mistrust,  over-reliance,  and 
complacency  (Billings  1997;  Parasuraman  and  Riley  1997;  Sarter  et 
al.  1997;  Sheridan  2002;  Wiener  1988). 

Parasuraman  et  al.  (2000)  proposed  that  these  unwanted  costs 
might  be  minimized  by  careful  consideration  of  different  informa¬ 
tion-processing  functions  that  can  be  automated.  Their  model  for 
effective  human-automation  interaction  design  identifies  four  stages 
of  human  information  processing  that  may  be  supported  by  auto¬ 
mation:  (stage  1)  information  acquisition;  (stage  2)  information 
analysis;  (stage  3)  decision  and  action  selection;  and  (stage  4)  action 
implementation.  Each  of  these  stages  may  be  supported  by  automa¬ 
tion  to  varying  degrees  between  the  extremes  of  manual  perfor¬ 
mance  and  full  automation  (Sheridan  and  Verplank  1978).  Because 
they  deal  with  distinct  aspects  of  information  processing,  the  first 
two  stages  (information  acquisition  and  analysis)  and  the  last  two 
stages  (decision  selection  and  action  implementation)  are  sometimes 
grouped  together  and  referred  to  as  information  and  decision  automa¬ 
tion,  respectively  (see  also  Billings  1997).  The  Theater  High  Alti¬ 
tude  Area  Defense  (THAAD)  system  used  for  intercepting  ballistic 
missiles  (Department  of  the  Army  2007)  is  an  example  of  a  fielded 
system  in  which  automation  is  applied  to  different  stages  and  at  dif¬ 
ferent  levels.  THAAD  has  relatively  high  levels  of  information 
acquisition,  information  analysis,  and  decision  selection;  however, 
action  implementation  automation  is  low,  giving  the  human  control 
over  the  execution  of  a  specific  action. 

Parasuraman  et  al.  (2000)  suggested  that  automation  could  be 
applied  at  very  high  levels  without  any  significant  performance  costs 
to  early-stage  functions,  i.e.  information  acquisition  and  analysis, 
particularly  if  the  automation  algorithms  used  were  highly  reliable. 
However,  they  suggested  that  for  high-risk  decisions  involving  con¬ 
siderations  of  lethality  or  safety,  decision  automation  should  be  set 
at  a  moderate  level  such  that  the  human  operator  is  not  only 
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involved  in  the  component  processes  involved  in  the  decisionmaking 
cycle,  but  also  makes  the  final  decision  (see  also  Wickens  et  al. 
1998).  As  noted  previously,  decision  aids  are  often  imperfect.  Sev¬ 
eral  empirical  studies  have  compared  the  effects  on  operator  perfor¬ 
mance  of  automation  imperfection  at  different  information- 
processing  stages.  Given  that  perfectly  (100  percent)  reliable  auto¬ 
mation  cannot  be  assured,  particularly  for  decisionmaking  func¬ 
tions,  it  is  important  to  examine  the  potential  costs  of  such 
imperfection.  Crocoll  and  Coury  (1990)  conducted  one  such  study. 
They  had  participants  perform  an  air-defense  targeting  (identifica¬ 
tion  and  engagement)  task  with  automation  support  that  was  not 
perfectly  reliable.  The  automation  provided  either  status  informa¬ 
tion  about  a  target  (information  automation)  or  a  recommendation 
concerning  its  identification  (decision  automation).  Crocoll  and 
Coury  found  that  the  cost  of  imperfect  advice  by  the  automation 
was  greater  when  participants  were  given  a  recommendation  to 
implement  (decision  automation)  than  when  given  only  status  infor¬ 
mation,  which  they  had  to  use  to  make  their  own  decision  (informa¬ 
tion  automation). 

A  more  extensive  study  by  Rovira  et  al.  (2007)  using  a  more  realistic 
simulation  confirmed  and  extended  these  findings.  They  examined 
performance  in  a  sensor-to-shooter  simulation  involving  a  battle¬ 
field  engagement  task  with  automation  support.  The  simulation 
consisted  of  three  components  shown  in  separate  windows  of  a  dis¬ 
play:  a  terrain  view,  a  task  window,  and  a  communications  module 
(see  Figure  1).  The  right  portion  of  the  display  showed  a  two-dimen¬ 
sional  terrain  view  of  a  simulated  battlefield,  with  identified  enemy 
units  (red),  friendly  battalion  units  (yellow),  artillery  units  (green), 
and  one  blue  friendly  headquarter  unit.  The  participants  made 
enemy-friendly  engagement  selections  with  the  aid  of  automation  in 
the  task  window.  They  were  required  to  identify  the  most  dangerous 
enemy  target  and  to  select  a  corresponding  friendly  unit  to  engage 
in  combat  with  the  target.  Like  Crocoll  and  Coury  (1990),  Rovira  et 
al.  (2007)  also  compared  information  and  decision  automation,  but 
examined  three  different  forms  of  decision  automation  to  examine 
the  generality  of  the  effect.  The  information  automation  gave  a  list 
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of  possible  enemy-friendly  unit  engagement  combinations  with 
information  (such  as  the  distance  between  units,  distance  from 
headquarters,  etc.),  but  left  the  decision  to  the  operator.  The  three 
types  of  decision  automation  gave  the  operator:  (1)  a  prioritized  list 
of  all  possible  enemy-friendly  engagement  choices;  (2)  the  top  three 
options  for  engagement;  or  (3)  the  best  choice  (these  were  termed 
low,  medium,  and  high  decision  automation,  respectively).  Also, 
importantly,  Rovira  et  al.  (2007)  varied  the  level  of  automation 
unreliability,  which  was  fixed  in  previous  studies.  The  decision  aid 
provided  recommendations  that  were  correct  80  percent  or  60  per¬ 
cent  of  the  time  at  the  high  and  low  levels  of  reliability.  This  manip¬ 
ulation  of  automation  reliability  allowed  for  an  assessment  of  the 
range  of  automation  imperfection  effects  on  decisionmaking,  as  well 
as  a  comparison  to  previous  work  on  complacency  in  which  auto¬ 
mation  reliability  was  varied  and  operator  reliance  on  automated 
aiding  was  evaluated  (Parasuraman  et  al.  1993). 


Figure  1.  Sensor  to  shooter  simulation  of  battlefield  engagement 
task,  with  automated  support  (Rovira  et  al.  2007). 
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Rovira  et  al.  (2007)  found  that  when  reliable,  automation  enhanced 
the  accuracy  of  battlefield  engagement  decisions.  This  benefit  was 
greater  for  decision  than  for  information  automation,  with  the  ben¬ 
efit  being  greater  as  the  level  of  decision  automation  (advisories) 
increased  from  low  (prioritized  list  of  all  possible  options)  to  medium 
(top  three  choices)  to  high  (best  choice)  (see  Figure  2).  More  impor¬ 
tantly,  reliable  decision  automation  significantly  reduced  the  time 
for  target  engagement  decisions.  This  is  important  for  shortening 
the  overall  sensor-to-shooter  time  in  tactical  C2  operations.  How¬ 
ever,  when  the  automation  provided  incorrect  assessments  on  occa¬ 
sion,  the  accuracy  of  target  engagement  decisions  at  those  times 
declined  drastically  for  all  three  different  levels  of  decision  automa¬ 
tion  (low,  medium,  and  high),  but  not  for  information  automation. 
The  accuracy  of  engagement  decisions  was  significantly  reduced 
(see  Figure  2)  when  incorrect  advisories  were  provided  by  the  auto¬ 
mation,  but  this  cost  was  obtained  only  for  decision  and  not  for 
information  automation. 


Info  Auto  Dec  Auto— Low  Dec  Auto— Medium  Dec  Auto— High 

Type  and  Level  of  Automation  Support 


Figure  2.  Decision  accuracy  under  different  forms  of  automation 
support  for  correct  and  incorrect  advisories  (Rovira  et  al.  2007). 
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These  results  confirm  the  previous  study  of  Crocoll  and  Coury 
(1990)  and  point  to  the  generality  of  the  effect  for  levels  of  decision 
automation  in  C2  systems.  Furthermore,  this  effect  was  particularly 
prominent  at  a  high  (80  percent)  overall  level  of  automation  reliabil¬ 
ity  and  less  so  at  a  lower  reliability  (60  percent).  This  finding  reflects 
one  of  the  paradoxes  of  decision  automation:  for  imperfect  (less 
than  1 00  percent)  automation,  the  greater  the  reliability,  the  greater 
the  chance  of  operator  over-reliance  (because  of  the  rarity  of  incor¬ 
rect  automation  advisories),  with  a  consequent  cost  of  the  operator 
uncritically  following  unreliable  advice.  These  results  also  demon¬ 
strate  that  the  cost  of  unreliable  decision  automation  occurs  across 
multiple  levels  of  decision  automation,  from  low  to  high. 

These  findings  may  be  interpreted  within  the  framework  of  the 
human-automation  interaction  model  proposed  by  Parasuraman  et 
al.  (2000).  Information  automation  gives  the  operator  status  infor¬ 
mation,  integrates  different  sources  of  data,  and  may  also  recom¬ 
mend  possible  courses  of  action,  but  not  commit  to  any  one  (Jones 
et  al.  2000).  However,  this  form  of  automation  typically  does  not 
give  values  to  the  possible  courses  of  action,  which  decision  automa¬ 
tion  does,  and  so  is  not  only  in  some  sense  “neutral”  with  respect  to 
the  “best”  decision,  but  also  does  not  hide  the  “raw  data”  from  the 
operator.  Thus  for  automation  that  is  highly  reliable  yet  imperfect, 
performance  is  better  with  an  information  support  tool  because  the 
user  continues  to  generate  the  values  for  the  different  courses  of 
action  and,  hence,  is  not  as  detrimentally  influenced  by  inaccurate 
information.  Additionally,  a  user  of  decision  automation  may  no 
longer  create  or  explore  novel  alternatives  apart  from  those  pro¬ 
vided  by  the  automation,  thus  leading  to  a  greater  performance  cost 
when  the  automation  is  unreliable. 

These  findings  suggest  that  in  high-risk  environments,  such  as  air 
traffic  control  and  battlefield  engagement,  decision  automation 
should  be  set  at  a  moderate  level  that  allows  room  for  operator 
input  into  the  decisionmaking  process.  Otherwise  there  is  a  risk  of 
operators  uncritically  following  incorrect  advisories  provided  by 
highly  but  not  perfectly  reliable  decision  support  systems.  At  the 
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same  time,  information  automation  can  assist  the  user  in  inference 
and  provide  information  relevant  to  possible  courses  of  action.  As 
Jones  et  al.  (2000)  have  demonstrated,  very  high  levels  of  automa¬ 
tion  can  be  effective  so  long  as  automated  help  is  confined  to  infor¬ 
mation  analysis  and  support  of  inference  generation  rather  than 
prescription  of  best  decision  outcomes. 


Adaptive  Automation  for  Human-Robot  Teaming 

Designing  automation  so  that  different  levels  of  automation  are 
applied  appropriately  to  different  stages  of  information  processing 
represents  one  method  to  ensure  effective  human-system  perfor¬ 
mance.  A  related  approach  is  to  vary  the  level  and  type  of  automa¬ 
tion  during  system  operations,  adaptively,  depending  on  context  or 
operator  needs.  This  defines  adaptive  or  adaptable  automation 
(Opperman  1994):  Information  or  decision  support  that  is  not  fixed 
at  the  design  stage  but  varies  appropriately  with  context  in  the  oper¬ 
ational  environment.  Adaptive /adaptable  automation  has  been 
proposed  as  a  solution  to  the  problems  associated  with  inflexible  or 
“static”  automation  (Inagaki  2003;  Parasuraman  and  Miller  2006; 
Scerbo  2001),  although  the  concept  has  also  been  criticized  as 
potentially  increasing  system  unpredictability  (Billings  and  Woods 
1994;  but  see  Miller  and  Parasuraman  2007). 

Adaptive  systems  were  proposed  over  20  years  ago  (Hancock  et  al. 
1985;  Parasuraman  et  al.  1992;  Rouse  1988),  but  empirical  evalua¬ 
tions  of  their  efficacy  have  only  recently  been  conducted  in  such 
domains  as  aviation  (Parasuraman  et  al.  1999),  air  traffic  manage¬ 
ment  (Hilburn  et  al.  1997;  Kaber  and  Endsley  2004),  and  industrial 
process  control  (Moray  et  al.  2000).  Practical  examples  of  adaptive 
automation  systems  include  the  Rotorcraft  Pilot’s  Associate.  This 
system,  which  aids  Army  helicopter  pilots  in  an  adaptive  manner 
depending  on  mission  context,  has  successfully  passed  both  simula¬ 
tor  and  rigorous  in-flight  tests  (Dornheim  1999).  In  the  context  of 
C2  systems,  adaptable  automation  allows  the  operator  to  define 
conditions  for  automation  decisions  during  mission  planning  and 
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execution  (Parasuraman  and  Miller  2006),  while  adaptive  automa¬ 
tion  is  initiated  by  the  system  (without  explicit  operator  input)  on 
the  basis  of  critical  mission  events,  performance,  or  physiological 
state  (Barnes  et  al.  2006). 

The  efficacy  of  adaptive  automation  must  not  be  assumed  but  must 
be  demonstrated  in  empirical  evaluations  of  human-robot  interac¬ 
tion  performance.  The  method  of  invocation  in  adaptive  systems  is 
a  key  issue  (Barnes  et  al.  2006).  Parasuraman  et  al.  (1992)  reviewed 
the  major  invocation  techniques  and  divided  them  into  five  main 
categories:  (1)  critical  events;  (2)  operator  performance  measure¬ 
ment;  (3)  operator  physiological  assessment;  (4)  operator  modeling; 
and  (5)  hybrid  methods  that  combine  one  or  more  of  the  previous 
four  methods.  For  example,  in  an  aircraft  air  defense  system,  adap¬ 
tive  automation  based  on  critical  events  would  invoke  automation 
only  when  certain  tactical  environmental  events  occur,  such  as  the 
beginning  of  a  “pop-up”  weapon  delivery  sequence,  which  would 
lead  to  activation  of  all  defensive  measures  of  the  aircraft  (Barnes 
and  Grossman  1985).  If  the  critical  events  do  not  occur,  the  automa¬ 
tion  is  not  invoked.  Hence  this  method  is  inherently  flexible  and 
adaptive  because  it  can  be  tied  to  current  tactics  and  doctrine  dur¬ 
ing  mission  planning.  This  flexibility  is  limited  by  the  fact  that  the 
contingencies  and  critical  events  are  anticipated.  A  disadvantage  of 
the  method  is  its  insensitivity  to  actual  system  and  human  operator 
performance.  One  potential  way  to  overcome  this  limitation  is  to 
measure  operator  performance  and/ or  physiological  activity  in  real 
time.  For  example,  operator  mental  workload  may  be  inferred  from 
performance,  physiological,  or  other  measures  (Byrne  and  Parasur¬ 
aman  1996;  Kramer  and  Parasuraman  in  press;  Parasuraman  2003; 
Wilson  and  Russell  2003).  The  measures  can  provide  inputs  to 
adaptive  logic  (which  could  be  rule  or  neural  network-based),  the 
output  of  which  invokes  automation  that  supports  or  advises  the 
operator  appropriately,  with  the  goal  of  balancing  workload  at  some 
optimum,  moderate  level  (Parasuraman  et  al.  1999;  Wilson  and 
Russell  2003). 
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Wiener  (1988)  first  noted  that  many  forms  of  automation  are 
designed  in  a  “clumsy”  manner,  increasing  the  operator’s  mental 
workload  when  his  or  her  task  load  is  already  high,  or  providing  aid 
when  it  is  not  needed.  An  example  of  the  former  is  when  an  aircraft 
flight  management  system  must  be  reprogrammed,  necessitating 
considerable  physical  and  mental  work  during  the  last  minutes  of 
final  descent,  because  of  a  change  in  runway  mandated  by  air  traffic 
control.  Thus,  one  of  the  costs  of  static  automation  can  be  unbalanced 
operator  workload.  There  is  good  evidence  that  adaptive  automa¬ 
tion  can  offset  this  and  balance  operator  mental  workload,  thereby 
protecting  system  performance  from  the  potential  catastrophic 
effects  of  overload  or  underload.  For  example,  Flilburn  et  al.  (1997) 
examined  the  effects  of  adaptive  automation  on  the  performance  of 
military  air  traffic  controllers  who  were  provided  with  a  decision  aid 
for  determining  optimal  descent  trajectories  of  aircraft — a  Descent 
Advisor  (DA)  under  varying  levels  of  traffic  load.  The  DA  was  either 
present  throughout,  irrespective  of  traffic  load  (static  automation), 
or  came  on  only  when  the  traffic  density  exceeded  a  threshold 
(adaptive  automation).  Flilburn  et  al.  found  significant  benefits  for 
controller  workload  (as  assessed  using  pupillometric  and  heart  rate 
variability  measures)  when  the  DA  was  provided  adaptively  during 
high  traffic  loads,  compared  to  when  it  was  available  throughout 
(static  automation)  or  only  at  low  traffic  loads. 

These  findings  clearly  support  the  use  of  adaptive  automation  to  bal¬ 
ance  operator  workload  in  complex  human-machine  systems.  This 
workload-leveling  effect  of  adaptive  automation  has  also  been  dem¬ 
onstrated  in  studies  using  non-physiological  measures.  Kaber  and 
Riley  (1999),  for  example,  used  a  secondary-task  measurement  tech¬ 
nique  to  assess  operator  workload  in  a  target  acquisition  task.  They 
found  that  adaptive  computer  aiding  based  on  the  secondary-task 
measure  enhanced  performance  on  the  primary  task.  The  results  of 
these  and  other  studies  indicate  that  adaptive  automation  can  serve  to 
reduce  the  problem  of  unbalanced  workload,  without  the  attendant 
high  peaks  and  troughs  that  static  automation  often  induces. 
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In  addition  to  triggering  adaptive  automation  on  the  basis  of  behav¬ 
ioral  or  physiological  measures  of  workload,  real-time  assessment  of 
situation  awareness  (SA)  might  also  be  useful  in  adaptive  systems 
(Kaber  and  Endsley  2004).  Reduced  SA  has  been  identified  as  a 
major  contributor  to  poor  performance  in  search  and  rescue  mis¬ 
sions  with  autonomous  robots  (Murphy  2000).  In  particular,  prob¬ 
ing  the  operator’s  awareness  of  changes  in  the  environment  might 
capture  transient  or  dynamic  changes  in  SA.  One  such  measure  is 
change  detection  performance.  People  often  fail  to  notice  changes  in 
visual  displays  when  they  occur  at  the  same  time  as  various  forms  of 
visual  transients  or  when  their  attention  is  focused  elsewhere 
(Simons  and  Ambinder  2005).  Real-time  assessment  of  change 
detection  performance  might  therefore  provide  a  surrogate  measure 
of  SA  that  could  be  used  to  trigger  adaptive  automated  support  of 
the  operator  and  enhance  performance.  Parasuraman  et  al.  (2007) 
examined  this  possibility  in  a  multi-task  scenario  involving  supervi¬ 
sion  of  multiple  UVs  in  a  simulated  reconnaissance  mission. 

Parasuraman  et  al.  (2007)  used  a  simulation  designed  to  isolate 
some  of  the  cognitive  requirements  associated  with  a  single  operator 
controlling  robotic  assets  within  a  larger  military  environment  (Bar¬ 
nes  et  al.  2006).  This  simulation,  known  as  the  Robotic  NCO  simu¬ 
lation,  required  the  participant  to  complete  four  military-relevant 
tasks:  (1)  a  UAV  target  identification  task;  (2)  a  UGV  (unmanned 
ground  vehicle)  route  planning  task;  (3)  a  communications  task  with 
an  embedded  verbal  SA  probe  task;  and  (4)  an  ancillary  task 
designed  to  assess  SA  using  a  probe  detection  method,  a  change 
detection  task  embedded  within  a  situation  map  (see  Figure  3).  The 
UAV  task  simulated  the  arrival  of  electronic  intelligence  hits 
(uelints”)  from  possible  targets  in  the  terrain  over  which  the  UAV 
flew.  When  a  target  was  detected,  it  was  displayed  in  the  UAV  view 
as  a  white  square.  Participants  were  told  that  when  a  target  was  pre¬ 
sented,  they  were  to  zoom  in  on  that  location,  which  opened  up  a 
window  of  the  UAV  view.  They  were  required  to  identify  the  target 
from  a  list  of  possible  types  (for  which  they  had  received  prior  train¬ 
ing).  Once  identified,  the  target  icon  was  then  displayed  on  the  situ¬ 
ation  map.  At  the  same  time  as  the  UAV  continued  on  its  flight  path 
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over  the  terrain,  the  UGV  moved  through  the  area  following  a 
series  of  pre-planned  waypoints  indicating  areas  of  interest  (AOI). 
During  the  mission,  the  UGV  would  stop  at  obstacles  at  various 
times  and  request  help  from  the  operator.  A  blocking  obstacle 
required  the  operator  to  re-plan  and  re-route  the  UGV,  whereas  a 
traversable  obstacle  required  that  the  participant  resume  the  UGV 
along  its  pre-planned  path.  Participants  received  messages  intermit¬ 
tently  while  performing  the  UAV  and  UGV  tasks.  The  communica¬ 
tions  task  involved  messages  presented  both  visually  in  a  separate 
communications  window  (see  Figure  3)  and  acoustically  over  a 
speaker.  The  messages  requested  updates  on  the  UGV/UAV  status 
and  the  location  of  particular  targets  (to  assess  SA,  as  described  fur¬ 
ther  below).  Participants  also  had  to  monitor  the  communications 
for  their  own  call  sign  (which  they  had  been  given  previously).  The 
communications  task  also  included  embedded  messages  designed  to 
assess  the  participants’  SA.  The  SA  questions,  which  were  divided 
into  two  types  according  to  Endsley’s  (1995)  taxonomy  of  SA — per¬ 
ception  and  comprehension — were  presented  during  the  mission 
and  required  a  Yes  or  No  response  from  the  participant. 


Figure  3.  Robotic  NCO  simulation. 


Parasuraman  et  al.  (2007)  examined  the  potential  efficacy  of  adap¬ 
tive  automation  based  on  real-time  assessment  of  operator  change 
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detection  performance,  on  performance,  situation  awareness,  and 
workload  in  supervising  multiple  UVs  under  two  levels  of  communi¬ 
cations  task  load.  They  used  an  adaptive  automation  invocation 
method  first  developed  by  Parasuraman  et  al.  (1996)  known  as  “per¬ 
formance-based”  adaptation.  In  this  method,  individual  operator 
performance  (i.e.,  change  detection  performance)  is  assessed  in  real 
time  and  used  as  a  basis  to  invoke  automation.  In  contrast,  in  static 
or  “model-based”  automation,  automation  is  invoked  at  a  particular 
point  in  time  during  the  mission  based  on  the  model  prediction  that 
operator  performance  is  likely  to  be  poor  at  that  time  (Parasuraman 
et  al.  1992).  This  method  is  by  definition  not  sensitive  to  individual 
differences  in  performance  since  it  assumes  that  all  operators  are 
characterized  by  the  model  predictions.  In  performance-based 
adaptive  automation,  on  the  other  hand,  automation  is  invoked  if 
and  only  if  the  performance  of  an  individual  operator  is  below  a  spec¬ 
ified  threshold  at  a  particular  point  in  time  during  the  mission.  If  a 
particular  operator  does  not  meet  the  threshold,  automation  is 
invoked.  However,  if  the  threshold  is  exceeded  in  another  operator, 
or  in  the  same  operator  at  a  different  point  in  the  mission,  the  auto¬ 
mation  is  not  invoked.  Thus,  performance-based  adaptation  is  by 
definition  context-sensitive  to  an  extent,  whereas  model-based  auto¬ 
mation  is  not. 

Parasuraman  et  al.  (2007)  examined  performance  under  three  con¬ 
ditions:  (1)  manual;  (2)  static  automation,  in  which  participants  were 
supported  in  the  UAV  task  with  an  automated  target  recognition 
(ATR)  system,  thereby  off-loading  them  of  the  responsibility  of  iden¬ 
tifying  targets;  and  (3)  adaptive  automation,  in  which  the  ATR  auto¬ 
mation  was  invoked  if  change  detection  accuracy  was  below  a 
threshold  (50  percent),  but  not  otherwise.  Each  of  these  conditions 
was  combined  factorially  with  two  levels  of  task  load,  as  manipu¬ 
lated  by  variation  in  the  difficulty  of  the  communications  task.  The 
results  were  broken  down  by  performance  in  the  pre-automation 
invocation  and  post-invocation  phases.  In  the  pre-invocation  phase 
before  automation  was  implemented,  there  were  no  significant  dif¬ 
ferences  in  change  detection  accuracy  between  conditions.  How¬ 
ever,  there  was  a  significant  effect  of  automation  condition  in  the 
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post-invocation  phase.  Participants  detected  significantly  more  icon 
changes  in  the  situation  map  in  the  static  and  adaptive  automation 
conditions  compared  to  manual  performance.  Furthermore,  change 
detection  performance  was  better  in  the  adaptive  automation  con¬ 
dition  than  in  the  static  automation  condition.  These  findings  were 
mirrored  in  the  analysis  of  the  verbal  SA  questions  that  probed  the 
operators’  awareness  of  the  relative  threat  imposed  by  enemy  units. 
Finally,  overall  operator  mental  workload,  as  assessed  subjectively 
(on  a  scale  from  0—100)  using  a  simplified  version  of  the  NASA- 
TLX  scale  (Flart  and  Staveland  1988),  was  lower  in  the  adaptive 
and  static  automation  conditions  compared  to  the  manual  condi¬ 
tion.  These  benefits  of  adaptive  automation  for  change  detection, 
SA,  and  workload  are  summarized  in  Figure  4. 
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Figure  4.  Effects  of  static  and  adaptive  automation  on  change  detec¬ 
tion  accuracy,  SA,  and  mental  workload.  From  Parasuraman  et  al. 
(2007). 
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Adaptive  or  Adaptable  Systems? 

The  results  of  the  study  by  Parasuraman  et  al.  (2007)  indicate  that 
both  overall  SA  and  mental  workload  were  positively  enhanced  by 
adaptive  automation  support  keyed  to  real-time  assessment  of 
change  detection  performance.  Coupled  with  the  previous  findings 
using  adaptive  automation  triggered  by  behavioral  and  physiologi¬ 
cal  measures  of  workload,  the  empirical  evidence  to  date  supports 
the  use  of  adaptive  automation  to  enhance  human-system  perfor¬ 
mance  in  systems  involving  supervision  of  multiple  UVs.  However, 
before  adaptive  systems  can  be  implemented  using  these  methods,  a 
critical  issue  must  be  addressed:  Who  is  “in  charge”  of  adaptation, 
the  human  or  the  system? 

In  adaptive  systems,  the  decision  to  invoke  automation  or  to  return 
an  automated  task  to  the  human  operator  is  made  by  the  system, 
using  any  of  the  previously  described  invocation  methods.  This 
immediately  raises  the  issue  of  user  acceptance.  Many  human  oper¬ 
ators,  especially  those  who  value  their  manual  control  expertise, 
may  be  unwilling  to  accede  to  the  “authority”  of  a  computer  system 
that  mandates  when  and  what  type  of  automation  is  or  is  not  to  be 
used.  Apart  from  user  acceptance,  the  issue  of  system  unpredictabil¬ 
ity  and  its  consequences  for  operator  performance  may  also  be  a 
problem.  It  is  possible  that  the  automated  systems  that  were 
designed  to  reduce  workload  may  actually  increase  it.  Billings  and 
Woods  (1994)  cautioned  that  truly  adaptive  systems  may  be  prob¬ 
lematic  because  the  system’s  behavior  may  not  be  predictable  to  the 
user.  To  the  extent  that  automation  can  hinder  the  operator’s  situa¬ 
tion  awareness  by  taking  him  or  her  out  of  the  loop,  unpredictably 
invoked  automation  by  an  adaptive  system  may  further  impair  the 
user’s  situation  awareness.  (There  is  evidence  to  the  contrary  from 
several  simulation  studies,  but  whether  this  would  also  hold  in  prac¬ 
tice  is  not  clear.) 

If  the  user  explicitly  invoked  automation,  then  presumably  system 
unpredictability  will  be  lessened.  But  involving  the  human  operator 
in  making  decisions  about  when  and  what  to  automate  can  increase 
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workload.  Thus,  there  is  a  tradeoff  between  increased  unpredict¬ 
ability  versus  increased  workload  in  systems  in  which  automation  is 
invoked  by  the  system  or  by  the  user,  respectively  (Miller  and  Para- 
suraman  2007).  These  alternatives  can  be  characterized  as  “adap¬ 
tive”  and  “adaptable”  approaches  to  system  design  (Opperman 
1984;  see  also  Scerbo  2001).  In  either  case,  the  human  and  machine 
systems  adapt  to  various  contexts,  but  in  adaptive  systems,  automa¬ 
tion  determines  and  executes  the  necessary  adaptations,  whereas  in 
adaptable  systems,  the  operator  is  in  charge  of  the  desired  adapta¬ 
tions.  Although  we  have  provided  evidence  for  the  efficacy  of  adap¬ 
tive  automation,  it  is  important  to  keep  in  mind  that  adaptable 
automation  may  provide  an  alternative  approach  with  its  own  bene¬ 
fits  (see  Miller  and  Parasuraman  2007).  In  adaptable  systems,  the 
human  operator  is  involved  in  the  decision  of  what  to  automate, 
similar  to  the  role  of  a  supervisor  of  a  human  team  who  delegates 
tasks  to  team  members,  but  in  this  case,  tasks  are  delegated  to  auto¬ 
mation.  The  challenge  for  developing  such  adaptable  automation 
systems  is  that  the  operator  should  be  able  to  make  decisions 
regarding  the  use  of  automation  in  a  way  that  does  not  create  such 
high  workload  that  any  potential  benefits  of  delegation  are  lost. 
There  is  some  preliminary  evidence  from  studies  of  human  supervi¬ 
sion  of  multiple  UVs  that  adaptable  automation  via  operator  dele¬ 
gation  can  yield  system  benefits  (Parasuraman  et  al.  2005). 
However,  much  more  work  needs  to  be  done  to  determine  whether 
such  benefits  would  still  hold  when  the  operator  is  faced  with  the 
additional  demands  of  high  workload  and  stress. 


Architectures  for  Adaptive  Automation 
in  Command  and  Control 

The  question  remains  whether  adaptive  or  adaptable  automation 
would  be  useful  in  a  specific  environment  representative  of  battle¬ 
field  C2  systems.  The  research  reported  in  this  paper  is  part  of  a 
broader  science  and  technology  program  aimed  at  understanding 
the  performance  requirements  for  human  robot  interactions 
(HRI)  in  future  battlefields  (Barnes  et  al.  2006).  Some  of  the  initial 
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results  have  shown  that  the  primary  tasks  that  soldiers  are 
required  to  perform  (and  will  continue  to  perform  in  the  future) 
place  severe  limits  on  their  ability  to  monitor  and  supervise  even  a 
single  UV,  let  alone  multiple  UAVs  and  UGVs.  The  results  of  a 
number  of  modeling  and  simulation  studies  indicate  that  the  sol¬ 
dier’s  primary  tasks  (e.g.,  radio  communications,  local  security), 
robotic  tasks  (e.g.,  mission  planning,  interventional  teleoperation), 
and  crew  safety  could  be  compromised  during  high  workload  mis¬ 
sion  segments  involving  HRI  (Chen  and  Joyner  2006;  Mitchell 
and  Henthorn  2005).  Some  variant  of  an  adaptive  system  would 
therefore  be  particularly  well  suited  to  these  situations  because  of 
the  uneven  workload  and  the  requirement  to  maintain  SA  for  the 
primary  as  well  as  the  robotic  tasks. 

What  adaptive  automation  architectures  should  be  considered? 
Based  on  the  empirical  results  discussed  previously,  we  suggest  that: 

1 .  Information  displays  should  adapt  to  the  changing  military 
environment.  For  example,  information  presentation  format 
(e.g,  text  vs.  graphics)  can  change  depending  on  whether  a  sol¬ 
dier  is  seated  in  a  C2  vehicle  or  is  dismounted  and  using  a  tablet 
controller. 

2.  Software  should  be  developed  that  allows  the  operator  to  allo¬ 
cate  automation  under  specified  conditions  before  the  mission  (as 
in  the  Rotorcraft  Pilot’s  Associate). 

3.  At  least  initially,  adaptive  systems  should  be  evaluated  that  do 
not  take  decision  authority  away  from  the  operator.  This  can  be 
accomplished  in  two  ways: 

(a)  an  advisory  asking  permission  to  invoke  automation,  or 

(b)  an  advisory  that  alerts  the  operator  that  the  automation  will 
be  invoked  unless  overridden. 

4.  For  safety  or  crew  protection  situations,  specific  tactical  or  safety 
responses  can  be  invoked  without  crew  permission. 

However,  it  will  be  important  to  evaluate  options  in  a  realistic  envi¬ 
ronment  before  final  designs  are  considered.  For  example,  perfor¬ 
mance-based  adaptive  logic  has  proven  to  be  effective  in  laboratory 


62  The  International  C2  Journal  |  Vol  1,  No  2 


simulations  but  may  not  be  practical  during  actual  missions  because 
of  the  temporal  sluggishness  of  performance  measures  (tens  of  sec¬ 
onds  to  many  minutes).  There  is  also  the  danger  of  adding  yet 
another  secondary  task  to  the  soldier’s  list  of  requirements  (although 
it  might  be  possible  to  assess  performance  on  “embedded”  second¬ 
ary  tasks  that  are  already  part  of  the  soldier’s  task  repertoire).  Physi¬ 
ological  measures  could  in  principle  be  used  to  invoke  adaptive 
automation  rapidly  (Byrne  and  Parasuraman  1996;  Parasuraman 
2003)  because  they  have  a  higher  bandwidth,  but  their  suitability  for 
rugged  field  operations  has  yet  to  be  demonstrated  reliably.  Also,  the 
type  of  tasks  to  automate  will  depend  on  engineering  considerations 
as  well  as  performance  and  operational  considerations.  Finally, 
there  is  the  system  engineering  challenge  of  integrating  different  lev¬ 
els  of  automation  into  the  overall  software  environment.  Fortu¬ 
nately,  there  are  simulation  platforms  that  capture  a  more  realistic 
level  of  fidelity  for  both  the  crew  tasking  environment  and  the  soft¬ 
ware  architecture  making  the  design  problem  more  tractable. 

The  future  promises  to  be  even  more  complex,  with  multiple,  highly 
autonomous  robots  of  varying  capabilities  being  supervised  by  a 
much  smaller  team  of  human  operators  who  will  only  be  alerted 
when  the  situation  becomes  critical  (Lewis  et  al.  2006).  However,  all 
the  performance  problems  discussed — over-reliance,  lack  of  situa¬ 
tion  awareness,  skill  degradation,  and  workload  increases  during 
crises — will,  if  anything,  worsen.  Automation  architectures  that  give 
operators  the  ability  to  adapt  to  the  current  situation  while  engag¬ 
ing  them  in  more  than  passive  monitoring  will  become  an  increas¬ 
ingly  important  part  of  the  HRI  design  process. 

Given  the  complexity  of  the  soldier's  environment,  the  multitude  of 
tasks  that  can  be  potentially  automated,  and  the  potential  human- 
performance  costs  associated  with  some  forms  of  automation,  how 
do  we  decide  not  only  how  to  implement  the  automation  but  also 
what  tasks  to  automate?  One  approach  we  are  pursuing  is  to 
develop  an  “automation  matrix”  that  prioritizes  proposed  tasks  for 
automation  for  UV  operations  (Gacy  2006).  The  matrix  provides 
the  framework  for  determining  what  is  automated  and  the  impact 
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of  such  automations.  Soldier  tasks  define  the  first  level  of  the  matrix. 
The  next  level  of  the  matrix  addresses  the  potential  automation 
approaches.  The  final  component  of  the  matrix  provides  weightings 
for  factors  such  as  task  importance,  task  connectedness,  and 
expected  workload.  A  simple  algorithm  then  combines  the  weights 
into  a  single  number  that  represents  the  overall  priority  for  develop¬ 
ment  of  that  automation.  From  this  prioritized  list,  various  automa¬ 
tion  strategies  (i.e.,  adaptive  or  adaptable)  can  be  evaluated.  We 
plan  further  analytical  and  empirical  studies  to  investigate  the  utility 
of  this  approach  to  human-automation  interaction  design  for  UV 
operations  in  C2  systems. 
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